---
title: "Overview"
description: "Automated Eval scoring, hallucination detection, bias monitoring for LLMs apps"
---

OpenLIT provides automated evaluation that helps you assess and monitor the quality, safety, and performance of your LLM outputs across development and production environments.

<video
      autoPlay
      muted
      loop
      controls
      className="w-full aspect-video rounded-xl"
      src="/images/evaluations.mp4"
    >
    </video>

## Why evaluations?

Evaluation is crucial for improving the accuracy and robustness of language models, ultimately enhancing user experience and trust in your AI applications. Here are the key benefits:

- **Quality & Safety Assurance**: Detect hallucinations, bias, toxicity, and ensure consistent, reliable AI outputs
- **Performance Monitoring**: Track model performance degradation and measure response quality across different scenarios
- **Risk Mitigation**: Catch potential issues before they reach users and ensure compliance with safety standards
- **Cost Optimization**: Monitor cost-effectiveness and ROI of different AI configurations and model choices
- **Continuous Improvement**: Build data-driven insights for A/B testing, optimization, and iterative development

## AI evaluation methods

OpenLIT provides automated LLM evaluation and testing capabilities for production AI applications:

### Automated LLM-as-a-Judge

Zero-setup AI quality monitoring that automatically evaluates your LLM responses:

- **Production Monitoring**: Auto-evaluate every LLM response for quality and safety issues
- **Smart Scheduling**: Configure evaluation frequency and sampling for cost optimization
- **Real-time Scoring**: Instant evaluation results visible in trace details and dashboards

### Programmatic evaluations  

Programmatic AI evaluation tools for custom testing workflows and development pipelines:

- **Code Integration**: Call LLM evaluators directly in your application code
- **CI/CD Quality Gates**: Automated testing for model improvements and regression detection

---

<CardGroup cols={3}>
  <Card title="LLM-as-a-Judge" href="/latest/openlit/evaluations/llm-as-a-judge" icon='gavel'>
    Use advanced LLMs to evaluate AI application quality with automated scoring
  </Card>
  <Card title="Programmatic evaluations" href="/latest/sdk/quickstart-programmatic-evals" icon='bolt'>
    Quick start guide for implementing custom evaluations in your code
  </Card>
  <Card title="Create a dashboard" href="/latest/openlit/dashboards/overview" icon='grid'>
    Create custom visualizations with flexible widgets, queries, and real-time AI monitoring
  </Card>
</CardGroup>